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INTRODUCTION 


Transforming educator evaluation systems, as with any large-scale reform, requires the support 
of stakeholders from a range of perspectives. Successful engagement contributes to making the 
most informed decisions, gaining the buy-in of key constituents, and increasing the likelihood of 
long-term sustainability. Sincere engagement of stakeholders requires compromise and honest, 
respectful dialogue, which can be challenging and necessitates a significant investment of time 
and resources. However, implementing reform without substantive stakeholder engagement could 
result in time and resources being lost as the reform falters from lack of support. This reality has 
been seen in a number of reform efforts where states or districts who experience success have 
engaged stakeholders throughout the process, and unsuccessful ones have experienced 
setbacks from a failure to obtain stakeholder input and support (Borman, Hewes, Overman, & 
Brown, 2004; Wallner, 2008). 

States must plan authentic efforts to engage teachers, principals, and the broader public in shaping 
key education policy decisions. Educator evaluation reform is more likely to lead to improved 
instruction and increased student learning when a cross-section of teachers and principals 
meaningfully participate in, and even lead, design and implementation efforts. Although state 
education agencies (SEAs) often embrace the importance of stakeholder engagement and 
communication, finding a practical, effective, and coherent means of accomplishing it remains 
an ongoing challenge. States need to take strategic and practical steps to capitalize on the 
expertise and valuable resources stakeholders offer. Therefore, states ought to consider 
multiple communication and engagement methods, with multiple audiences and at multiple 
times throughout the development and implementation process. 

With many efforts underway across the United States, SEA leaders have the opportunity to utilize 
the expertise of their contacts in other SEAs and regional comprehensive centers (RCCs) in their 
region and throughout the country to exchange ideas and share the lessons they have learned 
about involving stakeholders in meaningful and productive ways. To assist with this exchange, 
we provide a detailed example of one state’s experience with stakeholder engagement and 
communication efforts: Colorado. The National Comprehensive Center for Teacher Quality (TQ 
Center) selected Colorado after reviewing engagement and communication efforts in 24 states 
(see “The National Picture: How Do States Reach Out?”). Based on the variety and coordination 
of multiple communication strategies, Colorado emerged as a promising case to highlight. 


The National Picture: How Do States Reach Out? 
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To provide a snapshot of state-level efforts in reaching out and communicating with stakeholders, the TQ Center 
reviewed 24 states’ communication and engagement efforts around educator evaluation/ Although data on 
the quality and impact of these efforts are difficult to collect, it is possible to explore the multiple and 
varied methods of communication and stakeholder engagement that states have employed to date, as follows: 

• Websites. All of the states reviewed maintain a website related to the evaluation reforms. 

• Advisory committees. Most states (92 percent) have a state-level advisory committee that includes 
stakeholders as members. 

• News or social media. Only a handful of states have focused on news media (press conferences and news 
articles) or social media (13 percent); however, slightly less than half (42 percent) produce press releases 
highlighting their reforms. 

• Surveys. Less than half of the states reviewed (42 percent) used surveys (usually online) to collect 
stakeholder views. 

• Guidance and training material. Less than half of the states reviewed (42 percent) provided guidance, 
training material, and FAQ documents online. 

• Forums. Less than half (42 percent) hosted a series of regional forums to introduce reforms and/or hear 
stakeholders’ perspectives. 

• Presentations. SEAs and state-level committees used webinars or in-person presentations to LEAs, 
professional associations, teachers’ unions, or other stakeholder groups slightly less frequently (38 percent). 

• Online video or communication. One-third of states used additional online communication methods such 
as video or periodic e-newsletters to communicate information to stakeholders. 

• Focus groups. Focus groups are the least utilized in-person form of engagement, with 33 percent of states 
selecting this option as one means of gathering stakeholder views. 


^ Researchers drew on data from the TQ Center's State Teacher Evaluation Policy database (http://resource.tqsource. 
org/stateevaldb/) as well as additional reviews of SEA websites. The states reviewed included Arizona, Arkansas, 
Colorado, Delaware, Florida, Georgia, Idaho, Illinois, Indiana, Kentucky, Louisiana, Maryland, Massachusetts, Missouri, 
New York, Ohio, Oklahoma, Pennsylvania, Rhode Island, South Dakota, Tennessee, Washington, and Hawaii, as well 
as Washington, D.C. 


In designing its evaluation reforms, Colorado has taken a strategic approach to stakeholder 
communication and engagement. The key lessons that Colorado reform leaders have learned 
along their journey include the following: 

• Relationship building: prioritizing face-to-face interactions with stakeholders and 
approaching these as opportunities to genuinely listen to teachers and others. 

• Inclusiveness: involving all relevant parties, most particularly the statewide professional 
associations, at each stage of development and implementation. 

• Vision: keeping the vision central to the reforms to ensure that the SEA’s efforts and 
discussions with stakeholders stayed focused and avoided “vision creep,” in other words, 
losing sight of what they hope to accomplish and for what purposes. Referring back to this 
vision when obstacles arise is especially important, as is communicating to stakeholders 
concerning how the new policy will benefit them. 

• Expertise: recognizing the need to consult with experts, in addition to engaging with 
stakeholders. For example, as the scale of work demanded that the Colorado Department 
of Education educator effectiveness team expand from a close-knit group of three to nine, 
the group’s communication functions became more complex and the decision was made to 
hire a communication expert to assist with this task. A second expert consultant was hired 
to assist with rubric development. 


• Time: understanding the importance of allowing adequate time — although there may never 
be enough time — for each phase of the reform to be completed thoughtfully and in a way 
that allows for stakeholder engagement. 

Colorado invested significant time and resources into developing a comprehensive approach to 
stakeholder communication and engagement. Part of this approach included a strong emphasis 
on transparency, resulting in many of the state’s communication tools being freely available online 
for other SEAs to draw on in their own communication work (see “Online Access to Colorado Tools” 
for more information). 


Online Access to Colorado Tools 

The Colorado Department of Education (CDE) provides access to its process and communication tools for other 
SEAs to use and learn from. CDE Director of Educator Effectiveness Toby King notes that there is enough else to do 
around educator evaluation, so “don’t reinvent the wheel-take whatever you can from Colorado.” Information found 
at the following sites offers guidance: 

State Council for Educator Effectiveness (SCEE) 
http://www.cde.state.co.us/EducatorEffectiveness/Partner-SCEE.asp 

SCEE resources 

http://www.cde.state.co.us/EducatorEffectiveness/Partner-SCEE-Resources.asp 
Councils, boards, and partners 

http://www.cde.state.co.us/EducatorEffectiveness/Partners.asp 

Educator Effectiveness and Support (main page) 
http://www.cde.state.co.us/EducatorEffectiveness/index.asp 

Colorado’s Model Evaluation and Pilot information 

http://www.cde.state.co.us/EducatorEffectiveness/RB-CDEModelEvaluationSystem.asp 

Four-part video series on educator evaluation in Colorado 
http://www.cde.state.co.us/EducatorEffectiveness/index.asp 


We offer this Colorado case study to help clarify the key decisions, events, print and Web resources, 
and other specific communication approaches that SEAs and districts need to consider. In addition, 
this case study can be used as a companion piece to A Practical Guide to Designing Comprehensive 
Teacher Evaluation Systems (www.tqsource.org/publications/practicalGuideEvalSystems.pdf; 
Goe, Holdheide, & Miller, 2011), a tool that guides state planners through the eight components 
of teacher evaluation design. Component 2 of this tool. Securing and Sustaining Stakeholder 
Investment and Cultivating a Strategic Communication Plan, lays out the key considerations in four 
focus areas for ensuring stakeholder support for a new system. The four steps are (1) 
identifying the stakeholder groups, (2) establishing group roles and expectations, (3) 
developing a communication plan, and (4) soliciting feedback. The case study that follows is 
organized around Colorado’s activities and lessons learned under each of these four areas. The 
brief concludes with an overview of additional guidance and lessons learned from stakeholder 
engagement efforts in Teacher Incentive Fund districts and through work by the Bill & Melinda 
Gates Foundation, as well as an overview of existing resources on this topic area. 


COMMUNICATION AND STAKEHOLDER 
ENGAGEMENT IN COLORADO 


Background 

In 2010, the Colorado Legislature passed Senate Bill (S.B.) 10-191. Under this legislation, Colorado 
school districts have the option of selecting the state’s teacher evaluation model system or 
developing their own system by school year 2013-14. The legislation charged Colorado’s State 
Council on Educator Effectiveness (SCEE) with providing recommendations and guidelines to the 
State Board of Education (SBE) on a new system for evaluating teachers and principals. 

The various phases of the lawmaking process took place along the time frame shown in Table 1. 

Table 1. Educator Evaluation Reform Timeline 


Time Period 

Phase of Evaiuation Reform 

April-June 2011 

SCEE presented its recommendations to SBE 

June-November 2011 

SBE held three rule-making hearings and gathered written comments on 
the proposed rules 

November 2011 

SBE approved the final SB 191 rules and submitted them to the General 
Assembly for comment and revision 

May 2012 

Final approval of the rules 

August 2011-May 2015 

Piloting process 

August 2013 

Full statewide Implementation 


The policy specifies that by July 2013, each of Colorado’s 178 school districts will make publicly 
available its written teacher and principal evaluation systems and submit assurances to the 
Colorado Department of Education (CDE) that certain requirements are met. Included in the 
legislation is that: 

• Multiple performance measures are required. 

• Student growth measures must be included, and they must account for at least 50 percent 
of a teacher’s final evaluation score. Local education agencies (LEAs), however, may choose 
the student growth model they use. 

• Observation is a required feature of evaluations; however, no observation instrument is 
mandated by the state. 

• The state established minimum weights for each of the teaching standards, each of which 
must have a “measurable influence”; however, LEAs retain control over final decisions 
about how multiple measures should be aggregated into a final rating for a teacher. 

• Evaluation systems must have four levels of teacher performance. 

• The state requires that struggling teachers be provided support through professional 
development in order to improve within an established, reasonable time period. 




SECURING AND SUSTAINING STAKEHOLDER 
INVESTMENT AND CULTIVATING A STRATEGIC 
COMMUNICATION PLAN (TQ CENTER PRACTICAL 
GUIDE, COMPONENT 2) 

1. Identifying Stakeholder Groups 

When embarking on educator evaluation reform, a first step should be to identify the relevant 
stakeholder groups. Stakeholder groups should consist of individuals from across the education 
spectrum: from legislators to state agency staff to those with technical expertise in a relevant 
area to teachers, administrators, media, and local community members. Educators in particular 
bring an important perspective to evaluation design. They have unique insight into what works in 
the classroom and how to best evaluate their own work. This understanding is critical for the 
long-term success of a system that will support teaching and learning. Including educator voices 
also increases the likelihood that the changes will be perceived as responsive, fair, and beneficial, 
and ultimately that the changes will be accepted and promoted. Giving careful consideration to 
the members who make up stakeholder groups is an important first step to establishing an 
initiative that will be viewed as credible and trustworthy. The first step Colorado took was to create 
the State Council for Educator Effectiveness, a group that not only consisted of stakeholders 
appointed by the governor with input from the professional associations, but also brought in 
additional stakeholders (see step 2, “Establishing Group Roles and Expectations”). 

State Council for Educator Effectiveness (SCEE) 

Colorado’s commitment to stakeholder engagement began with the governor’s 15 appointed 
SCEE members. Among them, every stakeholder group was represented, with the teachers, 
through the teachers’ union, holding more positions than any other group. Specifically, SCEE 
consisted of the following stakeholders: 

• Two local school board members 

• Three teachers and one teachers union representative 

• Two principals 

• One student 

• One parent 

• One business member 

• One CDE representative 

• Two district administrators 

• One higher education faculty member 

In addition to identifying stakeholders for representation on the state council charged with 
designing the policy, Colorado identified the key statewide associations that should be at the 
table on an ongoing basis during the development and implementation process. Referred to 
as the “Five Cs," the following organizations were included: 


• Colorado Education Association 

• Colorado Association of School Boards 

• Colorado Association of School Executives 

• Colorado Department of Higher Education 

• Colorado Department of Education 

As is described further below, the “Five Cs" were in constant communication with one another 
and with the various stakeholder groups that these statewide associations represented. 

2. Establishing Group Roies and Expectations 

Stakeholders play a variety of roles in reform efforts. Different stakeholders engage in different 
ways for different purposes and carry different levels of responsibility. Reform leaders should set 
out clear expectations for each individual and group to avoid the pitfalls that can occur when 
expectations are mismatched. When determining roles and expectations, consider these factors: 

• What skills and experience does each stakeholder bring to the work? 

• What roles need to be filled? Where are there gaps in the group’s expertise? 

• What will the time commitment be? 

• Are some stakeholders representing another group, and thus should they be gathering 
feedback from them? 

• Which groups will serve as advisors and which as decision makers? 

SCEE faced many of these questions as council members tasked with guiding the process and 
making final recommendations to the State Board of Education. The complexity of the work required 
stakeholder subgroups in many areas to focus on particular aspects of evaluation design and 
implementation. To manage this process, SCEE created a network of advisory committees. 

Technical Advisory Groups 

In addition to the 15 stakeholders represented on the council, SCEE initiated more widespread 
engagement by setting up 11 technical advisory work groups (TAGs) to identify multiple measures 
of educator effectiveness, including multiple measures of student growth; evaluate effectiveness 
in light of the state requirements and local control issues; and create a report that would serve 
as the basis of SCEE’s recommendations in various mandated areas, including the following: 

• Teacher standards 

• Principal standards 

• Scoring framework and performance standards 

• Cost study 

• District guidelines 

• CDE guidelines for implementation and monitoring 

• Policy 

• Appeals process 

• Student growth 

• Parent and guardian involvement 

• Student involvement 


The TAGS included SCEE members as well as staff from education partner organizations in Colorado, 
CDE, The New Teacher Project, higher education faculty members, charter school representatives, 
American Federation of Teachers representatives, and even one state senator. 

Colorado Content Collaboratives 

Relying on teacher expertise in measuring student learning as well as providing another avenue 
for teacher engagement and ownership, CDE created Content Collaboratives; this initiative 
involves teachers in the creation of an assessment bank to measure student growth for teacher 
evaluations. CDE put out an open call for applications from all Colorado educators, and an 
internal team — along with external content and assessment professionals — blind-reviewed the 
applications to select participants. 

The Content Collaboratives convene educators from across the state to identify and create 
high-quality assessments, which are aligned to the new Colorado Academic Standards. The 
collaboratives are an ongoing initiative, with the first-year focus on creating fair, valid, and reliable 
measures of student learning that can be piloted and peer-reviewed for their utility in educator 
evaluations. In future years, using the measures, the collaboratives will focus on instructional 
practice and professional development to support effective practice. 

CDE also hired Jean Williams, Ph.D., Vice President for Research and Evaluation at Mid-Continent 
Research for Education and Learning, as an expert consultant to lead the development of the 
evaluation rubrics. In an interview, she noted that, even if an SEA has the expertise internally, 
convening groups such as the TAGs and Content Collaboratives helps to build credibility around 
the reform initiative. 

3. Developing a Communication Plan 

A strategic communication plan keeps the broader school community informed about the steps 
and progress of the reform. Portraying a clear and consistent message about implementation 
efforts, results, and future plans for implementation should be a top concern because it reduces 
room for misunderstanding and confusion. Thought needs to be given to these questions: 

• What are the main messages? 

• Who should hear these messages? 

• How would these messages be best delivered? 

Different groups may need different forms and levels of information, and it is important to 
understand those differences early. After these decisions are made, the most effective modes 
of communication can be linked with each audience and message type, and the best timing for 
information dissemination can be established. Colorado provides several examples of keeping 
key constituencies informed. 


CDE Key Communication and Engagement Strategies 


When Colorado was ready to launch its new policy, CDE adopted a number of strategies to 
communicate the new policy statewide. CDE Executive Director of Educator Effectiveness Katy 
Anthes explained, “The more communication you have, the better. Myths spread very quickly, so 
constant communication, including repetition of key facts, is critical.” Among the most important 
of Colorado’s communication strategies, according to CDE Director of Educator Effectiveness Toby 
King, was relationship building, both with stakeholders across Colorado and with other colleagues 
inside the SEA. 

Because teacher evaluation can be a hot-button issue. King explained that an important first step 
was hiring leaders at CDE with the right skills to interact with the field as an understanding 
partner. Colorado Education Association (CEA) Director of Teaching and Learning Linda Barker 
agreed: “The CDE team is very accessible, and credible in the field; they are very inclusive.” In 
interacting with stakeholders, CDE consistently reiterated two key messages: 

• The SEA does not have all the answers and wants to listen and learn from others in the field. 

• Continuous improvement is at the heart of this reform — for teachers, for districts, and for 
the SEA. 

Stakeholders reported to CDE that these messages represented a refreshing departure from the 
norm and built not only trust but also, in some cases, enthusiasm around the changes. CDE 
endeavored to speak and provide workshops and feedback sessions at all state association 
trainings. At the time that the evaluation rubrics were being developed, face-to-face focus groups 
were held during a statewide CEA conference. In addition, because of the strong relationships 
built between leaders at CDE and CEA, CEA extended an open invitation to all parties — including 
school board members, district staff, and others — to join the training sessions the SEA held for 
teachers on the new legislation. Through each of these activities, Colorado made efficient use 
of scarce resources by taking advantage of meetings that already were taking place. 

The state also prioritized limited resources by frontloading its efforts, focusing on building 
relationships early on in order to save time and resources later. In CDE’s communication efforts, 
face-to-face meetings were seen as important to building trust. Only after the relationships were 
built did CDE consider using webinars as a communication and engagement vehicle. 

The commitment to building trust applied to relationships both within and outside of CDE. 
The Educator Effectiveness unit determined that it was equally important to build internal 
relationships within the department itself. The two-pronged approach to building relationships 
proved an important component of CDE’s statewide communication efforts. 

Specifically, because Colorado was simultaneously changing its student performance standards 
and state assessments, among other policies, it was important to establish coherence in the 
messaging between the multiple initiatives so that teachers didn’t become overwhelmed. CDE’s 
Student Assessment, Teaching and Learning, and Educator Effectiveness units were in constant 
communication because of the interdependent nature of their work. Anthes explained: 


You can’t have one reform initiative without the other; they’re so interdependent that 
you couldn’t develop them separately or else you would have a very disjointed system. 
For example, new student standards aren’t helpful unless you have a new assessment 
system to measure progress toward those standards; and if you can’t measure progress 
toward those standards, then you can’t measure teacher effectiveness at teaching those 
standards. So we see it all as incredibly connected and, as such, could not silo the work 
We see it as core components of a system all coming together. 

The educator effectiveness team, in an effort to improve interdepartmental collaboration, led 
brown-bag lunches for all staff to share information and to consider alignment and messaging to 
stakeholders. As a result, the reform efforts and messages could be consistently communicated 
and questions answered at numerous statewide training sessions and meetings, which demonstrated 
a consistent, coherent message. At the very least, other SEA staff know with whom to put 
stakeholders in contact when questions arise. Breaking down these departmental silos will 
continue to benefit the SEA and the state in the future, both by helping each unit see the larger 
picture of education reform in the state and by creating efficiencies and greater coherence 
across initiatives. 

Colorado’s Communication Events 

Many of the strategies discussed here were described in SCEE’s Staying Informed document, 
which outlined for stakeholders all the ways in which CDE would be communicating and reaching 
out to the community, and various means for staying informed. 

Educator Effectiveness Symposia. To kick off communications about implementing the new 
legislation, in June 2011, the CDE Standards, Assessments, and Educator Effectiveness units 
Jointly convened four symposia across the state. Each was strategically held in locations with 
close proximity to districts. King explained, “We hadn’t realized the need to have us come to their 
territory." He also related that a useful strategy at these symposia was not only to communicate 
about the legislative changes but simultaneously to also recruit pilot districts. By recruiting pilot 
participants while they were most excited and most well informed about the reforms, CDE was 
able to attain twice as many pilots as needed. CDE decided to repeat these symposia again 
in June 2012. 

Educator Effectiveness Summit. Following the four initial symposia, a March 2012 statewide 
educator effectiveness summit was sponsored by CEA, Colorado Association of School Boards 
and Colorado Association of School Executives, in partnership with CDE, and hosted by the 
Colorado Legacy Foundation (a privately funded organization that serves as a “critical friend" and 
partner to CDE to help implement policy effectively). The one-day summit was attended by more 
than 500 participants from 94 school districts. It included presentations and opportunities for 
collaboration between district teams, experts, and pilot districts. Presentations by CDE Executive 
Director of Educator Effectiveness Katy Anthes and others provided an overview of Colorado’s 
educator evaluation policies, rules, and timelines. In addition, CDE distributed the following 
user-friendly resources: 


• Educator Evaluation System Implementation Toolkit that provides guidance to districts on 
how to prepare for implementation in 2013, including worksheets for assessing a district’s 
readiness for compliance with S.B. 191 

• A District Leader’s Guide: Communicating Effectively With Stakeholders that provides 
concrete advice and ideas to district leaders by drawing on the experiences and lessons 
learned from three Colorado districts that already had implemented new evaluation systems 

• A District Leader’s Guide: Preparing Your District to Implement a New Educator Evaluation 
System that provides district leaders an overview of the first steps they should take — 
establishing goals, assembling a design team, using guiding questions, and learning from 
peer districts — as well as tips and lessons learned from early implementer districts in Colorado 

• A Resource Bank Web page that hosts all information the districts need to begin the design 
and implementation process, including links to the state’s model evaluation system; 
sample evaluation systems from other districts or states; information on linking evaluation 
to a larger human capital management system; S.B. 191 rule making; frequently asked 
questions (FAQs); and wider resources on collective bargaining, evaluation, and compensation 
reform from national experts and technical assistance providers 

Written Communications Initiatives 

Features of the SEA’s Website. Just like many other states, Colorado’s communication and 
engagement strategies included a page on the CDE website specifically dedicated to the 
evaluation system reforms. Among other content, the website included the following: 

• A linked website that draws the connections between all of Colorado’s educator effectiveness 
initiatives, including recruitment, preparation, licensure and qualifications, induction, 
evaluation, and professional development and recognition 

• Regular reporting of meeting highlights 

• Background brochures and slides with overviews of S.B. 191 and SCEE 

• FAQs 

• Copies of all public comments, CDE responses, and multiple revisions to the rule-making 
documents, including documents that provide comparisons across different versions 

Educator Effectiveness Newsletter. CDE also launched the Educator Effectiveness E-Newsletter to 
provide regular updates (approximately every two months) on progress. The newsletter includes 
legislative background, updates, and timelines on changes to teacher evaluation in Colorado. All 
who are interested can sign up on the CDE’s Educator Effectiveness home page to receive the 
newsletter electronically. 

Communications With the “Five Cs” 

By keeping the website up-to-date, the newsletter frequent, and the channels of communication 
open, CEA and other statewide associations remained fully in the loop concerning developments 
to communicate to their members. Communication between Colorado’s Five Cs (see pp. 6-7), as 
well as the Colorado Legacy Foundation is a constant feature. Between meetings on various topics, 
the CDE educator effectiveness team interacts with the Five Cs in person about once every week 
or two, as well as weekly — indeed almost daily — contact by e-mail or telephone. At the time of this 


writing, the groups were in discussion about co-branding evaluation reform materials to ensure that 
a consistent and well-articulated message is presented to educators statewide. 

According to CEA Director of Teaching and Learning Linda Barker, the legislation could have been 
better written if all parties had been at the table. After the legislation had been passed, however, 
the union was able to become “a thoughtful partner.” All parties, and CDE leadership in particular, 
felt fortunate about establishing a positive relationship between CDE and CEA when the work 
began. Although CEA had opposed the initial legislation, after the bill passed, CEA leaders 
decided that rather than sitting on the sidelines, they would mobilize their members to make the 
reforms as smooth and as positive as possible. Indeed, all statewide stakeholder groups shared 
an “all-hands-on-deck” approach, understanding the need for everyone to work together to make 
the reforms work. 

It is important to note that the emphasis on engaging stakeholders began long before Colorado 
began bracing itself for implementation. The following description of the rule-making process 
illustrates how stakeholder communication and engagement were prioritized during that early 
stage of reform. 

Communications During Rule Making 

Colorado made specific, concrete plans for communicating with stakeholders and the public throughout 
the recommendations and rule-making process. The following are examples of plan execution: 

• During the rule-making stage, all written comments from stakeholders and the 
accompanying CDE responses were posted online. 

• CDE held three public meetings to listen to stakeholders during rule making, and SBE held 
three official rule-making hearings at which members of the public could testily. 

• CDE distributed a draft version of the rules before each of the three official public hearings, 
so that stakeholders could respond to the draft rules. 

4. Soliciting Feedback 

One of the critical roles stakeholders play is to provide informed feedback on a new evaluation 
model. Individuals and groups with technical expertise, educators and administrators are all 
constituents with unique perspectives who can provide invaluable feedback on a system’s 
reliability, fairness, feasibility, impact on practice, and more, all of which can be used to enhance 
system features. Colorado continues to go to great lengths to solicit feedback from across a wide 
range of stakeholder groups. Through the pilots, the Five Cs, conferences, focus groups, e-mail, 
webinars, the website, and other means, individuals and groups with technical expertise, 
educators, and administrators all provide unique perspectives that will strengthen the 
development and implementation of the new system. 

Statewide Stakeholder Input 

To solicit input on a wide scale, SCEE also engaged in the following outreach activities: 

• The council administered a survey to solicit input and advice on the proposed system; 
more than 1,750 individuals, mostly teachers, completed the survey. 


• CDE and the Colorado Legacy Foundation conducted 25 meetings across the state to 
discuss the recommendations, which included asking the more than 500 participants to (1) 
share their “best hopes and worst fears” about the new evaluation system and (2) 
contribute their advice and recommendations to the council. 

• All council meetings were publicized and open to the public, and each meeting’s agenda set 
aside time for attendees to give feedback to SCEE. At the meetings, 35 individuals and 
organizations gave public feedback, including parents, teachers, state legislators, CDE staff, 
school administrators, early childhood educators, superintendents, school board members. 
Board of Cooperative Educational Services representatives, business leaders, Colorado 
Education Association, Colorado Association of School Boards, and Colorado Association 
of School Executives. 

• The public also could submit comments directly via e-mail to the council, and these 
comments were sent directly to all council members. 

SCEE compiled the findings from these public input efforts in its Public Input and Survey Information 
Overview document that was posted online. Communicating back to stakeholders about how their 
input had been used, however, was an area in which interviewees agreed more could have been done. 

Focus Groups on Evaluation Rubrics. CDE invested in hiring Jean Williams, an expert rubric 
developer. To ensure that stakeholders were involved in designing the teacher and principal 
evaluation rubrics, Williams held many focus groups during a four-day CEA conference. On the 
final day of the conference, she presented to all participants the changes that were being made 
to the rubric as a result of their feedback. Thus, Williams used the focus groups not only to garner 
input from stakeholders, but also to demonstrate how their suggestions had been addressed. 

Williams drafted the initial rubrics on her own, but not without significant input from those who 
would be using the tools on the ground. Rather than selecting educators at random, she recruited 
a purposive sample of teachers and principals who were considered “deep thinkers" to provide 
assistance. Sixty teachers, including teacher leaders from CEA and the Center for Teacher Quality’s 
New Millennium Initiative, were involved in designing the teacher observation instrument. About 
30 of these teachers also had input into the principal evaluation rubric, and principals likewise 
weighed in both on the principal and teacher evaluation instruments. Teacher and principal input 
continues to be sought during the pilot process. 

Engagement and Feedback Through the Pilot Process. Although it required that Colorado push 
back full implementation, the state determined to conduct a three-year pilot of the new system, 
which includes 27 districts. The goals of the piloting process are to develop, identify, and/or 
test the following: 

• The teacher and principal evaluation rubrics 

• The measures of student academic growth 

• The methods of collecting teacher input for principal evaluations 

• The method of collecting student and family perception data 

• The method of aggregating measures and assigning final evaluation ratings 

• CDE monitoring methods 


Colorado is using the pilot process to collect stakeholder feedback about modifications to make 
to the system prior to statewide implementation, as well as to communicate CDE’s key message 
about the importance of continuous improvement in both the system itself and each individual — 
from policymakers to students — affected by it. 

In the view of Educator Effectiveness Executive Director Katy Anthes, a key engagement strategy 
during the initial rollout of these policies was to implement principal evaluations before teacher 
evaluations. Not only did that strategy help teachers see that their principals would be going 
through a similar evaluation process, but it also ensured principal engagement and understanding 
with these reforms, at scale. 


GUIDANCE FROM BEYOND COLORADO 


The Colorado example illustrates one approach to stakeholder engagement in educator evaluation 
design that is comprehensive in terms of identifying stakeholders, establishing group norms, 
communicating, and soliciting input. Communicating with and engaging stakeholders is not only 
best practice but also a required component of certain federal grant programs. In Race to the 
Top, for example, in order for a state to earn a high level of points within the State Success 
Factors section, it must demonstrate that it both provides and receives support from LEAs around 
the state — with a special mention of principals and teachers, including teachers unions. An 
Elementary and Secondary Education Act (ESEA) flexibility waiver requirement states: “Each SEA 
must engage diverse stakeholders and communities in the development of its request [for 
flexibility]” (U.S. Department of Education, 2012, p. 7). The Education Department highlights 
the need for input from a wide range of perspectives and asks SEAs applying for an ESEA waiver 
to demonstrate how they have requested feedback from diverse groups and how this information 
has influenced their planning. One section of its guidelines states directly, “To receive this 
flexibility, an SEA and each LEA must commit to develop, adopt, pilot, and implement, with the 
involvement of teachers and principals, teacher and principal evaluation and support systems” 
(U.S. Department of Education, 2012, p. 6). This requirement underscores the importance of 
meaningfully including those most impacted by performance evaluations. 

Other education reform incentive programs also have led to learning around stakeholder 
communication and engagement. For example, teacher-performance-related pay reform efforts 
preceded the current wave of teacher evaluation reforms and offer lessons to the latter. Those 
involved in these compensation reforms as Teacher Incentive Fund (TIF) grantees have learned 
a number of important lessons regarding stakeholder engagement and communication that can 
be useful beyond TIF initiatives. Julia Koppich (2010) sums up these lessons in Meeting the 
Challenges of Stakeholder Engagement and Communication: Lessons from Teacher Incentive Fund 
Grantees. Specifically, she describes two groups of stakeholders: internal and external. Internal 
stakeholders are those who are most directly affected by the reforms. New evaluation plans 
affect teachers and principals more than any other groups. External stakeholders are those 
who may have an interest in the reforms but are less directly impacted; examples include 
parents, the media, and business and community leaders. The internal-external distinction is 
important because each group requires different communication and engagement approaches 
in order for their valuable contributions to be made. The following lessons from TIF grantees 
about communicating and engaging with internal and external stakeholders are useful to 
evaluation system developers: 

• Identify from the beginning who the main stakeholders are, internal and external. 

• Invite internal stakeholders into the process early on, and continually engage with them 
throughout implementation. 

• Develop a comprehensive, multistrategy communication plan that will communicate efforts 
throughout planning, development, and implementation. 

• Reach out for help when challenges arise. Chances are that others have faced similar 
issues regarding engagement and communication (Koppich, 2010). 


The Bill & Melinda Gates Foundation, in promoting new teacher evaluation reforms, lends further 
support for the importance of stakeholder communication and engagement. In 2009, the 
foundation requested that nine school districts and an association of charter school management 
companies develop plans for transforming their evaluation systems. To guide the participants, 
the foundation laid out a few principles that drew on the expertise of district leaders, school 
administrators, and teachers identifying best practices in teacher evaluation. Among the lessons 
that the school districts learned was the importance of stakeholder engagement during planning 
and implementation, and the creation and execution of a multidimensional communication plan. 

In fact, all 10 participants highlighted this as a key recommendation for others embarking on similar 
reforms (Bill & Melinda Gates Foundation, 2010). Recognizing the value of stakeholder perceptions, 
the Gates Foundation has since begun funding a “teacher voice” initiative to support more than 
a dozen grassroots efforts across the country working to better engage teachers in the 
policymaking process. 


Resources to Support Effective Stakeholder Engagement 

Marketing and communication are generaiiy not an area of expertise with most educators, especiaiiy in smaiier 
districts; however, severai resources are avaiiabie that provide guidance and ideas regarding how to effectiveiy 
communicate with and engage stakehoiders at every stage of deveiopment.The Nationai Comprehensive Center 
forTeacher Quaiity (TQ Center) created A Practical Guide to Designing Comprehensive Teacher Evaluation Systems 
and A Practical Guide to Designing Comprehensive Principal Evaluation Systems. These guides are toois intended 
to faciiitate discussion around the important components of system design, of which communication and 
stakehoider engagement is one. They present questions to work through that heip states and districts ensure 
that criticai considerations are not being ieft out. 

A reiated resource from the TQ Center is the Communication Framework for Measuring Teacher Quality and 
Effectiveness: Bringing Coherence to the Conversation. This brief describes a four-component framework that 
promotes productive diaiogue around teacher effectiveness. Finaiiy, theTQ Center has aiso created State Teacher 
Evaluation Policies (STEP) and State Principal Evaluation Policies (SPEP), databases of state poiicies and 
practices reiated to the eight design components of teacher and principai evaiuation. Using STEP and SPER 
educators and poiicymakers can review the state-ievei poiicies currentiy enacted in states across the country. 

The Center for Educator Compensation Reform (CECR) was estabiished by the U.S. Department of Education 
in 2006 to support Teacher incentive Fund (TIF) grantees, poiicymakers, states, districts, and schoois with toois, 
information, and other resources in their move toward performance-based compensation systems. Because 
incentive-based compensation and evaiuation are so cioseiy iinked, educators and poiicymakers designing 
these systems experience many of the same chaiienges. CECR provides research, state exampies, and detaiied 
information on data systems, student growth measures, and program evaiuation. The center aiso suggests 
concrete ideas for communicating strategicaiiy and buiiding stakehoider support through a six-step guide 
to impiementation. 

Everyone at the Table: Engaging Teachers in Evaluation Reform is an initiative of American institutes for Research 
and Pubiic Agenda, with funding from the Biii & Meiinda Gates Foundation. 

This free oniine resource center provides an easy-to-use modei for widespread teacher-ied conversations on 
evaiuation reform that are constructive and soiutions oriented, using structured conversation toois and activities, 
with the end goai of increasing teacher input into the poiicies that are deveioped. it inciudes: 

• A two-minute video that captures the importance and enthusiasm of education ieaders around the country 
for broader, more genuine invoivement of teachers in evaiuation reform (http://www.everyoneatthetabie. 
org/ieadersVideo.php) 

• An eight-minute teacher discussion-starter video (http://www.everyoneatthetabie.org/gtt_video.php) that 
gives teachers the chance to think and taik about the pros and cons of different kinds of evaiuation 
systems. 

• Materiais such as moderator's guides, PowerPoint presentations, and discussion summary tempiates to 
heip ieaders organize discussions with teachers and bring their voices to the tabie. 

Everyone at the Tabie has been used with success in LosAngeies, Detroit, Washington state, and eisewhere.To read 
their stories and iearn more about this innovative approach to teacher engagement around evaiuation, visit www. 
everyoneatthetabie.org. 


CONCLUSION 


Colorado has strived for accessible, credible, and inclusive stakeholder engagement, bringing 
in a broad representation of internal and external stakeholders. This strategy and the state’s 
commitment to transparency and regular and clear communication in a variety of formats have 
combined to lay the foundation for strong stakeholder involvement, increasing Colorado’s chances 
for a sustainable educator evaluation system. Although the process has not been without 
obstacles, leaders in the effort have faced these challenges as collaborators working toward the 
same ends, not as adversaries. It is this “all-hands-on-deck” attitude that has allowed Colorado 
to move forward in promising ways. 

Although each state has its own unique context, many experiences and challenges are shared, a 
situation that positions states to be helpful resources to each other. Much can be borrowed from 
the Colorado example and adapted to fit other states’ contexts. Key ideas include a clear and 
consistent message of continuous improvement — a standard the leaders applied to themselves 
as well as to schools and educators — and face-to-face communication whenever feasible until 
relationships are firmly established. Colorado strategically hired new staff to fill expertise gaps 
within the organization and devoted the time necessary for each phase of planning and 
implementation. Colorado’s lessons apply to any state context, and the practices detailed in 
this brief are intended to provide other SEAs and RCCs with concrete examples that can be 
adapted and modified to meet contextual needs. 

To spark these initiatives, states should communicate and engage with one another to share 
their successes, challenges, and innovative ideas. In addition, Web resources such as those 
mentioned in this brief support states and districts in their efforts for meaningful engagement 
and are relevant at every stage of reform — design, planning, implementation, and evaluation of 
the system — and at every level — state, district, and school. Although earlier is always better, 
it is never too late to bring invested groups into the process and authentically engage them. 
Leveraging the knowledge, ideas, and concerns of your state’s educators whenever possible 
is a cost-effective investment. Investing in stakeholders could mean the difference between a 
system with little stakeholder support that does not meet the professional needs of educators 
and a robust, enduring system that all educators believe will make a measurable difference in 
teaching and learning. 
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